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15.  Supplementary  Notes 


16.  Abstract 

This  final  report  documents  the  tests  results  on  the  Airport  Movement 
Area  Safety  System  (AMASS)  Air  Traffic  (AT)  Operational  Testing  conducted  at 
Atlanta,  GA  and  Detroit,  MI  International  Airports.  The  purpose  of  the  AMASS 
Operational  Test  (OT)  was  to  verify  the  Critical  Operational  Issues  (COI) . 

The  AMASS  is  a  runway  collision  alert  system  that  provides  tower  air 
traffic  controllers  with  automated  conflict  warnings  and  alerts  to 
reduce  the  risks  of  runway  collisions.  The  AMASS  system  provides  tower 
controllers  with  both  aural  and  visual  alerts. 

The  AMASS  system  receives  raw  video  from  the  Airport  Surface  Detection  Equipment-3 
( ASDE-3 )  and  airborne  approach  targets  from  the  Terminal  Automation  Interface  Unit 
(TAIU) .  The  TAIU  receives  target  data  from  the  Surveillance  Communications 
Interface  Processor  (SCIP)  and  aircraft  tag  data  from  the  Automated 
Radar  Tracking  System  (ARTS) .  It  tracks  this  data  and  sends  appropriate 
position,  vector,  and  a  predicted  approach  runway  to  AMASS.  Targets 
are  then  processed  by  the  AMASS  Safety  Logic  to  determine  if  any 
targets  have  the  potential  to  create  a  hazardous  situation. 

The  Operational  Testing  on  the  AMASS  COIs  questions  has  been  evaluated/assessed, 
resulting  in  no  high  issues.  The  Airway  Facilities  (AF) /AT  OT  Test  Team  members  felt 
that  the  AMASS  system  has  shown  dramatic  improvements  and  is  currently 
operationally  ready. 
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EXECUTIVE  SUMMARY 


The  Operational  Test  (OT)  for  the  Airport  Area  Movement  Safety  System  (AMASS)  was  split 
into  two  testing  phases,  Airway  Facilities  (AF)  OT  and  Air  Traffic  (AT)  OT.  The  AF  OT  testing 
was  conducted  at  Detroit  Metropolitan  Wayne  Airport  (DTW)  on  April  20  through  29,  1999.  The 
AT  OT  testing  was  conducted  at  William  B.  Hartsfield  Atlanta  Airport  (ATL)  on  July  18  through 
August  20, 1999.  Any  open  issues  that  resulted  from  the  AF  and  AT  OT  testing  were  verified 
during  AF  and  AT  regression  testing.  The  AF  OT  regression  testing  was  conducted  at  DTW  and 
was  held  on  February  22  and  23,  2000.  The  AT  OT  regression  testing  was  conducted  at  ATL  on 
June  1  through  23,  2000. 


The  Operational  Testing  on  the  AMASS  Critical  Operational  Issues  (COI)  questions  have  been 
evaluated/assessed,  resulting  in  no  high  issues.  The  following  medium  issues  are  associated  with 
COI  listed  below: 

COI  1 :  Does  the  AMASS  provide  the  controller  with  alerts  on  potential  runway  collisions? 

AT  Issue  8:  (Medium)  AMASS  is  currently  limited  to  12,000  feet  of  movement  area 
coverage  from  the  antenna. 

Resolution:  Modify  AMASS  to  support  the  full  Airport  Surface  Detection  Equipment-3 
(ASDE-3)  range  coverage  of  up  to  24,000  feet  (System  Trouble  Report  (STR)  has  been 
written). 

Status:  This  problem  will  be  fixed  prior  to  Dallas  Fort  Worth  International  Airport 
(DFW)  and  Denver  International  Airport  (DEN)  going  Operational  Readiness 
Declaration. 

COI  2:  Is  the  AMASS  site  adaptable  (e.g.,  eliminate  unwanted  alerts  due  to:  multipath, 
sequential  alerts,  etc.)? 


By  implementing  key  software  fixes  the  false  alert  rate  was  reduced  from  over  60  per 
month  to  approximately  4  per  month.  The  nuisance  alert  rate  that  coincided  with  the 
AMASS  final  parameter  set  was  approximately  two  per  month. 


AF  Issue  2:  (Medium)  Line  Replacement  Unit  (LRU)  replacement  from  the  front  of  the 
cabinet  for  the  Central  Processing  Unit  (CPU)  is  very  difficult.  Cable  dressing 
complicated  the  removal/replacement  procedures,  with  cable  ties  having  to  be  cut  to 
facilitate  removal  of  equipment. 
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Resolution:  Modify  AMASS  equipment  to  support  LRU  replacement  from  the  front  of 
the  cabinet. 

Status:  STR  has  been  written. 


COI  3:  Does  the  AMASS  provide  the  controls  and  displays  necessary  for  the 
controller/supervisor  to  perform  the  assigned  duties? 

AT  Issue  6a:  (Medium)  AT  Training  should  include  a  clear  and  concise  presentation  on 
the  AMASS  Alert  Scenarios. 

Resolution:  Provide  a  video  demonstrating  each  AMASS  alert  scenario  on  the  potential 
collisions. 

Status:  Currently  the  training  does  provide  slides  on  each  alert  scenario,  but  the  OT  test 
team  members  want  to  see  a  video  demonstration  on  each  AMASS  alert  scenario  (Visual 
Aid  on  the  general  timing  and  distance  of  each  alert  scenario). 


AT  Issue  7b:  (Medium)  The  AT  Users  Manual  provides  too  much  information  for  use  by 
a  controller  on  a  day-to-day  basis. 

Resolution:  Include  a  Quick  Reference  Card  to  summarize  the  AMASS  commands  (in 
addition  to  the  planned  Mini-Display  Control  Unit  (DCU)  keypad  template.) 

Status:  AMASS  Work  Group  has  this  issue  and  is  in  the  process  of  deciding  whether  or 
not  to  develop  a  quick  reference  card. 


COI  4:  Does  the  AMASS  provide  the  controller  the  capability  to  inhibit/re-inhibit  specific  targets 
from  activating  alerts? 

No  concems/issues  were  expressed  with  the  inhibit/re-inhibit  functionality. 


COI  5:  Does  the  AMASS  degrade  any  other  FAA  system  to  which  it  is  interfaced? 

AF  Issue  10:  (Medium)  AMASS  is  generating  a  video  stitching  on  the  ASDE-3  Display. 
Resolution:  Modify  the  AMASS  equipment  to  resolve  the  video  timing  issue. 

Status:  A  screen  refresh  removes  the  video  stitching;  this  problem  will  be  fixed  (STR  has 
been  written). 


The  AF/AT  OT  Test  Team  members  felt  that  the  AMASS  system  has  shown  dramatic 
improvements  and  is  currently  operationally  ready. 


1. 


INTRODUCTION. 


1.1  PURPOSE. 

The  purpose  of  Airport  Movement  Area  Safety  System  (AMASS)  Operational  Testing  was  to 
verify  that  the  AMASS  satisfies  the  five  following  Critical  Operational  Issues  (COI)  defined  in 
the  AMASS  Operational  Requirements  Document  (ORD): 

COI-1 :  Does  the  AMASS  provide  the  controller  with  alerts  on  potential  runway  collisions? 

COI-2:  Is  the  AMASS  site  adaptable  (e.g.,  eliminate  unwanted  alerts  due  to:  multipath, 

sequential  alerts,  etc.)? 

COI-3:  Does  the  AMASS  provide  the  controls  and  displays  necessary  for  the 

controller/supervisor  to  perform  the  assigned  duties? 

COI-4:  Does  the  AMASS  provide  the  controller  the  capability  to  inhibit/re-inhibit  specific 

targets  from  activating  alerts? 

COI-5:  Does  the  AMASS  degrade  any  other  Federal  Aviation  Administration  (FAA)  system 

to  which  it  is  interfaced? 

To  accomplish  the  above  task,  the  AMASS  Operational  Test  (OT)  Plan  allocated  each  individual 
AMASS  ORD  requirement  to  one  or  more  of  the  five  COIs  as  Measures  of  Performance  (MOP). 
A  success  criteria,  or  Minimum  Acceptable  Operating  Requirement  (MAOR),  was  then  derived 
from  each  MOP.  The  AMASS  Operational  Test  Plan  then  allocated  each  MOP/MAOR  pair  to 
either  the  Airways  Facilities  (AF)  OT  effort  or  the  Air  Traffic  (AT)  OT  effort.  Each 
MOP/MAOR  pair  allocated  to  the  AT/AF  OT  effort  was  then  further  allocated  to  one  or  more  of 
the  three  planned  AT/AF  OT  test  procedures. 


1.2  SCOPE. 

This  report  discusses  the  results  of  the  AMASS  OT  testing;  the  OT  testing  was  split  into  two 
testing  phases,  an  AF  and  AT.  The  AF  OT  testing  was  conducted  at  Detroit  Metropolitan  Wayne 
Airport  (DTW)  on  April  20  through  29, 1999.  The  AT  OT  testing  was  conduct  at  William  B. 
Hartsfield  Atlanta  Airport  (ATL)  on  July  1 8  through  August  20, 1 999.  The  open  issues  that 
resulted  from  the  AF  and  AT  OT  testing  were  verified  during  an  AF  and  AT  regression  testing. 
The  AF  OT  regression  testing  was  conducted  at  DTW  and  was  held  on  February  22  and  23, 
2000.  The  AT  OT  regression  testing  was  conducted  at  ATL  on  June  1  through  23,  2000. 
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2.  REFERENCE  DOCUMENTS. 


The  following  documents  are  referred  in  this  test  report  or  provide  useful  information: 

a.  AMASS  Operational  Test  Plan,  dated  March  1999 

b.  AMASS  AF  OT  Test  Procedures,  dated  March  1999 

c.  AF  OT  Quick  Look  Report,  dated  April  1 999 

d.  AF  OT  Regression  Quick  Look  Report,  dated  February  1 6,  2000, 

Report  Number:  DOT/FAA/CT/ACT300-00-02 

e.  AMASS  OT  AT  Test  Procedures,  dated  May  1999 

f.  AT  OT  Quick  Look  Report,  dated  June  1999 

g.  AMASS  OT  AT  Regression  Test  Plan/Procedures,  dated  June  12,  2000 

h.  AT  OT  Regression  Quick  Look  Report,  dated  June  23,  2000, 

Report  Number:  DOT/FAA/CT/ACT3 00-00-01 


3.  AMASS  SYSTEM  DESCRIPTION. 

3.1  MISSION  REVIEW. 

The  AMASS  is  a  runway  collision  alert  system  that  provides  tower  air  traffic  controllers  with 
automated  conflict  warnings  and  alerts  to  reduce  the  risks  of  runway  collisions.  The  AMASS 
system  provides  tower  controllers  with  both  aural  and  visual  alerts. 


3.2  AMASS  SYSTEM  DESCRIPTION. 


The  AMASS  system  receives  raw  video  from  the  Airport  Surface  Detection  Equipment-3 
(ASDE-3)  and  airborne  approach  targets  from  the  Terminal  Automation  Interface  Unit  (TAIU). 
The  TAIU  receives  target  data  from  the  Surveillance  Communications  Interface  Processor 
(SCIP)  and  aircraft  tag  data  from  the  Automated  Radar  Tracking  System  (ARTS).  It  tracks  this 
data  and  sends  appropriate  position,  vector,  and  a  predicted  approach  runway  to  AMASS. 
Targets  are  then  processed  by  the  AMASS  Safety  Logic  to  determine  if  any  targets  have  the 
potential  to  create  a  hazardous  situation.  When  it  is  determined  there  is  a  possible  hazardous 
situation,  AMASS  will  deliver  an  aural  alert,  a  visual  text  message,  and  highlight  the  targets  in 
potential  conflict. 
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3.3  TEST  SYSTEM  CONFIGURATION. 


AT  Operational  Testing 

The  AMASS  was  tested  with  full  functionality  except  for  the  Pre-Product  Improvements.  The 
AMASS  system  relies  on  the  ASDE-3/TAIU  configuration,  local  airport  operations,  and  external 
equipment  interfaces.  All  the  AMASS  alerts  were  tested  in  the  factory,  but  only  a  subset  was 
optimized  for  the  various  sites.  The  test  was  conducted  using  the  AMASS  software  Build  2.0 
and  included  local  site  adaptation  parameters  at  ATL.  The  TAIU  software  was  Build  2.0  and 
ASDE-3  software  Build  61 1 .3. 

The  airport  configuration  at  ATL  consisted  of  four  parallel  runways.  The  ASDE-3  antenna  is 
located  on  top  of  the  tower  cab.  The  ASDE-3/ AMASS  is  configured  as  a  seven-channel  system. 
The  ATL  Terminal  Radar  Approach  Control  Facility  (TRACON)  is  co-located  at  the  airport  and 
also  the  ARTS  IHA.  Terminal  Surveillance  data  is  provided  by  ASR-9  located  at  the  airport.  The 
AMASS  TAIU  is  located  at  the  TRACON. 

AF  Operational  Testing 

The  AMASS  was  tested  with  full  maintenance  functionality.  The  AMASS  system  relies  on  the 
ASDE-3  RMS  interface.  The  test  was  conducted  using  the  AMASS  software  Build  1 .0  and 
included  local  site  adaptation  parameters  at  DTW.  The  TAIU  software  was  Build  1 .0  and 
ASDE-3  software  Build  611.3. 

The  DTW  ASDE-3  antenna  is  located  on  top  of  the  tower  cab.  The  ASDE-3/AMASS  is 
configured  as  a  five-channel  system.  The  DTW  TRACON  is  co-located  at  the  airport  and  also 
the  ARTS  IHA.  Terminal  Surveillance  data  is  provided  by  ASR-9  located  at  the  airport.  The 
AMASS  TAIU  is  located  at  the  TRACON. 

3.4  INTERFACES. 

The  AMASS  external  interfaces  consist  of  the  ASDE-3,  and  the  TAIU.  The  TAIU  interfaces 
with  the  SCEP,  and  the  ARTS  IHA. 

The  ASDE-3  provides  radar  target  data  on  objects  up  to  the  tower  height,  and  24,000  feet 
horizontal.  The  ASDE-3  also  provides  the  display/control  functions,  and  the  RMS  interface  to 
the  AMASS  via  serial  lines.  For  airborne-approach  aircraft,  surveillance  radar  data  is  received 
from  the  SCIP  via  the  AMASS  TAIU  and  transmitted  to  the  AMASS  cabinet  by  serial  modems. 
The  TAIU  also  interfaces  with  the  ARTS  IHA  to  receive  flight  plan  information. 

Only  the  ARTS  IHA,  ARTS  IHE,  and  ARTS  HE  interfaces  are  currently  developed.  The 
remaining  automation  interfaces  such  as  Enroute  Automated  Radar  Tracking  System  (pEARTS), 
and  Standard  Terminal  Automation  Replacement  System  (STARS)  have  yet  to  be  developed  for 
the  TAIU.  These  preplanned  interface  improvements,  along  with  flight  plan  functionality, 
including  the  use  of  aircraft  tag  data,  were  not  evaluated  during  this  Operational  Testing. 
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4.  TEST  AND  EVALUATION  DESCRIPTION. 


The  OT  testing  was  split  into  two  testing  phases;  AF  and  AT.  The  AT  OT  testing  was  broken 
down  into  three  testing  events;  Developmental  Test  (DT)  Data  Review,  AMASS  Parameter  Test, 
and  the  Normal  Operations  Test.  The  AF  OT  testing  was  broken  down  into  two  testing  events; 
DT  Data  Review,  and  the  System  Maintenance  Activities  Test.  The  following  sections  discuss 
the  test  events/results  of  the  AMASS  AT/AF  OT  testing. 


4. 1  AT  OT.  DT  DATA  REVIEW. 

4.1.1  Test  Event  Description. 

The  AT  DT  Data  Review  was  performed  at  the  Technical  Center.  The  DT  Data  Review  consisted 
of  a  desk  audit  of  the  contractor  performed  Development  Test  and  Site  Acceptance  Test  reports, 
System  Trouble  Reports  (STR),  the  AMASS  ORD  and  the  AMASS  Specification  (FAA-E- 
2869),  the  AMASS  AT  training  course,  document  reviews  and  formal  contractual  approvals. 

This  review  evaluated  whether  each  of  the  MOPs,  identified  in  the  DT  procedure,  had  been 
satisfactorily  tested  and  whether  the  results  met  the  corresponding  success  criteria  (MAORs). 


4. 1 .2  Results/Discussion. 

All  the  MOPs/MAORs  passed  except  for  MOP/MOAR  1-26  listed  below.  Appendix  A  to  this 
report  includes  the  MOP/MAOR  Result  Matrix. 


MOP-1.26: 

Clutter  Map.  AMASS  shall  maintain  a  clutter  map  to  aid  in  the  removal  of  static  or  slowly 
varying  clutter. 

MAOR-1.26: 

An  AMAS  clutter  map  must  be  maintained  to  aid  in  the  removal  of  static  or  slowly  varying 
clutter. 


MOP/MOAR  1-26  is  failed  because  AMASS  Clutter  Map  is  limited  to  12,000  feet  of  movement 
area  coverage  where  the  ASDE-3  area  coverage  is  24,000  feet.  Norden  will  modify  the  AMASS 
system  to  support  the  full  ASDE-3  range  coverage  (STR  has  been  written).  This  issue  (AT  Issue 
#8,  appendix  B)  was  classified  as  a  Medium  and  will  be  fixed  prior  to  Dallas  Fort  Worth  Airport 
(DFW)  and  Denver  International  Airport  (DEN)  going  Operational  Readiness  Declaration. 


Appendix  B  to  this  report  includes  the  AT  Issue  Matrix. 
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4.2  AT  QT.  AMASS  PARAMETER  TESTING. 


The  purpose  of  the  AMASS  Parameters  Testing  was  to  determine  if  the  AMASS  could  be 
adapted  to  allow  for  the  implementation  of  the  alert  set  and  parameterization  concept  desired  by 
the  AMASS  Working  Group  (AWG).  This  test  event  was  divided  into  three  sub-tests;  the 
AMASS  Parameter  Study  (APS),  the  Operational  Events  Test  (OET)  and  the  Operational  Usage 
Test  (OUT).  These  test  events  were  conducted  at  the  Technical  Center  using  the  ACT-310 
AMASS  laboratory  facilities. 


4.2. 1  AMASS  Parameter  Study  (APS). 


4.2.1 .1  Test  Event  Description. 

The  purpose  of  the  APS  sub-test  was  to  determine  if  the  AMASS  is  sufficiently  adaptable  to 
minimize  false  and  nuisance  alerts  and  to  develop  an  initial  alert  set  based  on  parameterization 
concept  provided  by  the  AMASS  Working  Group. 


4.2. 1.2  Data  Collection/ Analysis. 

The  process  began  with  the  collection  of  AMASS  data  at  San  Francisco  International  Airport 
(SFO),  ATL,  and  DTW.  More  than  1000  hours  of  AMASS  data  was  recorded  in  SFO  and  ATL, 
and  was  sent  to  the  Technical  Center  for  analysis.  Additionally,  approximately  500  hours  of 
usable  data  were  collected  in  DTW.  Data  were  collected  for  AMASS  using  both  the  STANDBY 
and  ONLINE  ASDE-3  transmitters.  In  addition,  both  encoder  A  and  B  were  exercised  during 
data  collection. 


The  analysis  consisted  of  replaying  the  recorded  data  utilizing  the  playback  feature  of  the 
AMASS.  This  feature  allows  for  data  to  be  played  using  different  alert  sets,  time/distance  alert 
parameters,  airport  configurations,  and  multipath  parameters.  The  initial  analysis  of  the  data 
received  from  Atlanta  began  with  an  alert  and  parameter  set  that  was  conservative.  Alert  listings 
were  generated  and  analyzed  to  determine  the  number  of  false  alerts  (primarily  due  to  false 
targets)  and  real  alerts.  Once  this  analysis  was  completed,  the  AMASS  parameters  were  modified 
in  an  attempt  to  reduce  the  number  of  nuisance  alerts  that  did  not  meet  the  guidelines  set  forth  in 
the  AMASS  Working  Group.  This  process  was  repeated  through  four  subsequently  tighter  alert 
sets  until  the  parameterization  of  the  AMASS  alert  set  was  optimized  to  minimize  nuisance 
alerts. 


4.2. 1,3  Results/Discussion. 

All  the  MOPs/MOARs  (appendix  C)  in  this  sub-test  passed  with  no  open  issues. 

The  init  adaptation  test  had  six  STRs  written  to  lower  the  false  alert  rate.  The  main  issues  found 
during  this  test  are: 
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a.  (Issue  #le,  appendix  B)  False  tracks  can  reach  real  track  status  on  a  taxiway  and  then 
move  across  the  grass  and  onto  runway  surfaces  resulting  in  false  alerts. 

b.  (Issue  #ld,  appendix  B)  AMASS  generates  false  tracks  and  false  alerts  in  heavy  rain. 

c.  (Issue  #la,  appendix  B)  Long-term  false  tracks  on  a  runway  surface  will  transition  to 
“real”  track  status  within  4  minutes. 


These  issues  listed  above  were  verified/passed  during  the  AT  OT  regression-testing  period.  The 
false  alert  rate  was  reduced  from  over  60  per  month  to  approximately  4  per  month.  The  nuisance 
alert  rate  that  coincided  with  the  final  parameter  set  was  approximately  two  per  month.  The  final 
alert  parameter  study  was  presented  to  the  AMASS  Work  Group  on  June  23, 2000,  at  the  OT 
Regression  testing  phase,  and  the  final  parameter  listing  is  contained  in  appendix  D. 


4.2.2  Operational  Events  Test  (OET). 


4.2.2. 1  Test  Event  Description. 

The  purpose  of  the  OET  sub-test  was  to  demonstrate  the  AMASS  response  to  each  alert  situation 
included  in  the  core  alert  set.  The  OET  sub-test  utilized  the  data  collected  and  analyzed  during 
the  APS  sub-test  to  develop  specific  scenarios  for  each  alert  cell  that  was  identified  in  the  AWG 
alert  set.  Twenty-eight  test  cases,  comprised  of  one  or  more  iterations,  were  developed. 


4.2.2.2  Data  Collection/ Analysis. 

AT  controllers  were  asked  to  view  each  alert  scenario  and  record  how  the  AMASS  response  to 
the  situation  presented  would  affect  their  ability  to  perform  their  assigned  duties  when  following 
the  Draft  ATC  AMASS  Operational  Procedures.  The  test  team  responses  were  then  transcribed 
and  reviewed  (appendix  E).  Test  team  discussions  were  held  to  review  the  composite  results.  The 
feedback  received  on  the  questionnaires  and  in  the  discussion  sessions  were  then  used  to 
determine  if  any  further  modifications  to  the  initial  parameterization  concept  were  necessary 
prior  to  the  conclusion  of  the  test  and  in  preparation  for  OT  at  ATL. 


4.2.2.3  Results/Discussion. 

The  execution  of  the  OET  sub-test  resulted  in  minor  modifications  to  the  time/distance 
parameterization  of  AMASS.  All  the  MOPs/MOARs  (appendix  F)  in  this  sub-test  have  passed 
with  no  open  issues. 


One  of  the  main  issues  found  during  this  test  relates  to  taxiing  aircraft  across/off  the  runway 
surfaces,  which  sometimes  can  cause  alerts  because  these  tracks  do  not  include  a  projection 
vector.  The  AMASS  system  was  modified  to  use  a  projection  vector  which  reduced  the  number 
of  nuisance  alerts.  These  issues  were  verified  and  closed  at  the  OT  regression-testing  period. 
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4.2.3  Operational  Usage  Test  (OUT)  Sub-Test. 


4.2.3 .1  Test  Event  Description. 

The  purpose  of  the  Operational  Usage  Test  (OUT)  was  to  examine  the  AT  training, 
documentation,  and  usage  of  the  AMASS  User  Interface. 


4.2.3.2  Data  Collection/Analysis. 

The  OUT  sub-test  was  performed  after  the  initial  AT  OT  training  introduction  was  given  and 
after  the  test  team  members  had  an  opportunity  to  use  the  AMASS  user  interface.  A 
questionnaire  (appendix  G)  was  given  to  each  test  team  and  a  discussion  session  was  held  to 
review  the  answers  and  any  issues. 


4.2. 3. 3  Results/Discussion. 

All  the  MOPs/MOARs  (appendix  H)  in  this  sub-test  were  passed  during  the  AT  OT  regression¬ 
testing  period. 


Several  issues  were  generated  during  this  testing  phase.  The  two  issues  that  remain  open  are  the 
following: 

a.  (Issue  #6b,  Medium,  appendix  B)  AT  Training  should  include  a  clear  and  concise 
presentation  (video  demonstration)  on  the  AMASS  Alerts. 

b.  (Issue  #7b,  Medium,  appendix  B),  the  test  team  members  felt  that  the  AT  User’s  Manual 
was  too  cumbersome  for  day-to-day  use  and  that  a  Quick  Reference  Card  should  be 
included  in  the  manual. 

4.3  OPERATIONAL  TESTING  AT  ATL. 


The  purpose  of  performing  operational  testing  at  the  ATL  was  to  expose  the  test  team  members 
to  AMASS  operations  in  an  operational  setting.  The  operational  testing  was  divided  into  two 
sub-tests;  the  OET  and  OUT. 


4.3.1  Operational  Events  Test  (PET)  Sub-Test. 


4.3. 1.1  Test  Event  Description. 

The  OET  sub-test  consisted  of  injecting  synthetic  targets  against  Targets  of  Opportunity  to  create 
each  alert  scenario  script.  The  OT  Test  Team  members  would  evaluate  each  alert  scenario  script 
in  the  Tower  Cab. 


4.3. 1.2  Data  Collection/ Analysis. 


The  test  team  was  asked  to  view  each  alert  scenario  and  record  how  the  AMASS  response  to  the 
situation  presented  would  affect  their  ability  to  perform  their  assigned  duties  when  following  the 
Draft  ATC  AMASS  Operational  Procedures.  The  feedback  received  on  the  questionnaires 
(appendix  I)  and  in  the  discussion  sessions  were  then  used  to  determine  if  any  further 
modifications  to  the  AMASS  parameters  were  necessary  prior  to  the  conclusion  of  the  test. 


4.3.1 .3  Results/Discussion. 

The  execution  of  the  OET  sub-test  resulted  in  minor  modifications  to  the  time/distance 
parameterization  of  AMASS.  All  open  issues  in  this  sub-test  have  passed  and  was  verified 
during  the  AT  OT  regression-testing  period. 


One  of  the  issues  during  this  test  was  the  TAIU  might  not  provide  the  first  update  for  a  new 
target  under  track  until  up  to  4  seconds  after  entering  the  runway  approach  zone.  The  TAIU  was 
fixed  to  provide  the  first  update  within  1  second  after  entering  the  runway  approach  zone.  This 
issue  was  verified  and  retested  at  the  AT  OT  regression-testing  period. 


4.3.2  Operational  Usage  Test  (OUT)  Sub-Test. 


4.3.2. 1  Test  Event  Description. 

The  purpose  of  this  OUT  sub-test  was  to  examine  the  training,  documentation,  and  usage  of  the 
AMASS  User  Interface  in  an  operational  setting. 


4.3 .2 .2  Data  Collection/ Analysis. 

Test  team  members  were  given  the  opportunity  to  use  the  AT  User’s  Manual  and  AMASS  User 
Interface  to  perform  the  three  AMASS  operations  prior  to  completing  the  questionnaires 
(appendices  J  and  K)  which  included  selecting  operational  configurations,  opening/closing 
runways,  and  adjusting  the  alert  speaker  volume.  The  AT  Test  Team  members  had  formal 
AMASS  AT  training  prior  to  going  to  ATL. 


4.3 .2.3  Results/Discussion. 

All  open  issues  in  this  sub-test  passed  and  were  verified  during  the  AT  OT  regression-testing 
period  except  for  issues  6b  and  7b  (refer  to  section  4. 2. 3. 3). 
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4.4  AF  OT.  DT  DATA  REVIEW. 


4.4,1  Test  Event  Description. 

The  AF  DT  Data  Review  was  performed  at  the  Technical  Center.  The  DT  Data  Review  consisted 
of  a  desk  audit  of  the  contractor-performed  Development  Test  and  Site  Acceptance  Test  reports, 
STRs,  the  AMASS  ORD  and  the  AMASS  Specification  (FAA-E-2869),  the  AMASS  AT  training 
course,  document  reviews,  and  formal  contractual  approvals.  This  review  evaluated  whether  each 
of  the  MOPs  identified  in  the  DT  procedure  had  been  satisfactorily  tested  and  whether  the  results 
met  the  corresponding  success  criteria  (MAORs). 


4.4.2  Results/Discussion. 

All  the  MOPs/MAORs  have  passed.  Appendix  L  to  this  report  includes  the  MOP/MAOR  Result 
Matrix. 


4.5  MAINTENANCE  ACTIVITIES  TEST. 

4.5.1  Test  Event  Description. 

The  AMASS  AF  Operational  Testing  took  place  April  20  through  29, 1999,  at  DTW.  This  test 
was  concerned  with  determining  whether  the  AMASS  (including  contractor  provided  training 
and  documentation)  is  effective  and  suitable  for  inclusion  into  the  National  Airspace  System 
(NAS)  from  a  maintenance  perspective.  The  evaluation  approach  used  was  to  perform  a  series  of 
test  procedures  with  test  teams  comprised  of  representative  users  from  AF  field  organizations. 
Three  AF  technicians  were  in  attendance  representing  Central,  Great  Lakes,  and  Southern 
regions.  The  test  team  members  were  presented  with  a  series  of  test  procedures,  each  having  a 
defined  objective.  After  each  test  procedure  was  performed,  the  test  team  members  filled  out  a 
series  of  questionnaires. 


Three  types  of  testing  were  conducted  at  the  DTW.  These  were: 

a.  System  Corrective  Actions  Test 

b.  System  Maintenance  Activities  Test 

c.  NAS  Integration  and  Degraded  Operations  Test 

The  AF  System  Corrective  Actions  Test  verified  that  AF  Technicians  have  the  tools,  training, 
and  documentation  necessary  to  maintain  the  AMASS  to  the  Line  Replacement  Unit  (LRU) 
level.  This  test  was  performed  using  AF  Technicians  to  detect,  isolate,  and  correct  faults  inserted 
into  the  system  by  the  test  engineers.  Opinions  were  collected  from  the  AF  Technicians  as  to  the 
correctness  and  completeness  of  the  provided  materials  to  accomplish  the  assigned  tasks  at  the 
end  of  each  test  script. 
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The  System  Maintenance  Activities  Test  verified  that  the  AF  Technicians  have  the  tools, 
training,  and  documentation  necessary  to  ensure  the  AMASS  system  is  operating  correctly  on  a 
day-to-day  basis.  This  includes  the  documentation,  procedures,  and  test  equipment  necessary  to 
certify  the  system  for  use  on  a  day-to-day  basis.  It  also  includes  a  verification  of  the  logistics 
support  system  in  place  to  support  the  AMASS  system  throughout  its  life  cycle.  This  includes 
logistical  support  as  it  relates  to  the  LRU,  consumables,  software  maintenance,  site  adaptation 
and  optimization,  and  other  logistical  support  activities  included  in  the  AMASS  Maintenance 
concept.  This  test  used  AF  Technicians  to  perfonn  the  specified  maintenance  activities  included 
in  the  Maintenance  Handbook.  They  were  asked  to  provide  feedback  on  the  acceptability  of  the 
provided  training,  equipment,  and  related  materials. 


The  AMASS  NAS  Integration  and  Degraded  Operations  Test  verified,  from  an  AF  perspective, 
that  the  requirements  associated  with  the  three  interfaces  to  the  systems  are  already  established  in 
the  NAS  system.  These  interfaces  are:  the  AMASS/TAIU,  the  AMASS/ASDE-3  display 
subsystem,  and  the  AMASS/ASDE-3  Remote  Maintenance  Monitoring  (RMS)  Subsystem.  The 
test  also  verified  the  ability  of  the  AMASS  to  continue  to  perform  the  assigned  mission  when  the 
AMASS  system  is  not  operating  at  100  percent.  This  testing  took  place  from  two  perspectives. 
First,  it  focused  on  the  ability  of  the  AMASS  to  continue  operating  when  one  of  its  subsystem 
has  failed.  Failures  included  both  AMASS  implemented  interface  failures  and  core  LRU 
failures.  Secondly,  the  ability  of  the  AMASS  to  continue  to  perform  its  mission  when  the 
ASDE-3  is  operating  in  various  degraded  states.  (This  section  was  verified  using  Maintenance 
Demo  held  at  Norden.) 

Dimensions  International,  developers  of  the  TAIU,  provided  the  TAIU  operation  and 
familiarization  training  as  well  as  the  preventive  maintenance  procedure  overview.  The  training 
was  extremely  useful,  as  none  of  the  technicians  were  familiar  with  the  TAIU  system  before 
arrival.  The  technicians  cited  the  TAIU  users  and  troubleshooting  manuals,  as  being  acceptable, 
although  clarification  is  required  in  many  areas.  Many  of  the  concerns  arose  over  the  areas  of 
logistic  and  second  level  support.  Detailed  training  at  each  site,  as  proposed,  will  remedy  these 
concerns.  Appendix  M  to  this  report  includes  the  composite  AF  questionnaires  (February  1 999). 


4.5.2  Results/Discussion. 

In  April  1999,  the  major  issue  was  that  the  AMASS  manuals  were  inadequate  in  performing 
LRU  replacement  and  general  maintenance  procedures  on  the  AMASS  system.  AOS  discussed 
system  certification  issues  with  the  test  participants  at  length,  and  began  development  of 
certification  procedures. 


All  open  issues  except  for  issue  #2  and  #10  were  verified  and  passed  during  the  regression 
testing  at  DTW  the  week  of  June  18,  2000.  The  AF  technicians  performed  certification 
procedures  and  reviewed  the  change  pages  to  the  Maintenance  manual.  The  AMASS  manuals 
were  improved  but  additional  comments  were  generated  by  the  AF  technicians  and  were 
incorporated.  The  technicians  followed  the  certification  procedures  step-by-step.  There  are  some 
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discrepancies  that  need  to  be  corrected,  but  the  technicians  feel  they  can  certify  the  AMASS  and 
TAIU. 


Issue  #2  remains  open  because  the  LRU  replacement  from  the  front  of  the  cabinet  for  the  Central 
Processor  Unit  (CPU)  is  very  difficult.  Cable  dressing  complicated  the  removal/replacement 
procedures,  with  cable  ties  having  to  be  cut  to  facilitate  removal  of  equipment.  This  issue 
(appendix  N)  was  classified  as  a  Medium.  Issue  #10  is  open  because  AMASS  is  generating  a 
video  stitching  on  the  ASDE-3  Display.  This  issue  was  classified  as  a  medium  because  the  video 
stitching  occurs  only  on  a  few  displays  and  can  take  hours  and  even  days  to  become  visible;  also 
any  type  of  screen  refresh  on  the  display  removes  the  video  stitching. 


Appendix  N  to  this  report  includes  the  AF  Issue  Matrix. 

Appendix  O  to  this  report  includes  the  AF  Regression  Test  comments  to  the  AMASS  manuals. 


5.  ISSUE  ASSESSMENT. 


The  OT  test  team  analyzed  the  correlated  test  data  to  determine  if  operational  issues  existed  and 
if  so,  assessed  the  operational  impact.  Each  issue  was  identified  a  criticality  which  has  been 
defined  as  follows: 


a.  High  -  A  problem  that  will  prevent,  degrade,  or  interrupt  operational  service  or  jeopardize 
safety,  and  has  no  acceptable  work-around. 

b.  Medium  -  To  prevent,  degrade,  or  interrupt  operational  service  or  jeopardize  safety,  but 
has  an  acceptable  work-around. 

c.  Low  -  The  issue  constitutes  an  improvement  to  the  operational  use  of  AMASS  and  can  be 
resolved  through  post-commissioning  modifications. 


6.  COI  RESULTS. 


The  evaluation  results  specific  to  each  COI  are  presented  in  the  following  subsections  and 
categorized  by  the  MOP/MAOR.  All  the  MOPs/MAORs  are  listed  in  appendices  A  and  L. 


6.1  COIl: 

Does  the  AMASS  provide  the  controller  with  alerts  on  potential  runway  collisions? 
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6.1.1  C0I1  Results. 


The  following  issue  is  associated  with  this  COI: 


AT  Issue  8:  (Medium)  AMASS  is  currently  limited  to  12,000  feet  of  movement  area  coverage 
from  the  antenna. 

Resolution:  Modify  AMASS  to  support  the  full  ASDE-3  range  coverage  of  up  to  24,000  feet 
(STR  has  been  written). 

Status:  This  problem  will  be  fixed  prior  to  DFW  and  DEN  going  Operational  Readiness 
Declaration. 


6.2  COI  2: 

Is  the  AMASS  site  adaptable  (e.g.,  eliminate  unwanted  alerts  due  to:  multipath,  sequential  alerts, 
etc.)? 


6.2.1  COI  2  Results. 

The  following  issues  are  associated  with  this  COI: 


AF  Issue  2:  (Medium)  LRU  replacement  from  the  front  of  the  cabinet  for  the  CPU  is  very 
difficult.  Cable  dressing  complicated  the  removal/replacement  procedures,  with  cable  ties  having 
to  be  cut  to  facilitate  removal  of  equipment. 

Resolution:  Modify  AMASS  equipment  to  support  LRU  replacement  from  the  front  of  the 
cabinet. 

Status:  STR  has  been  written. 


6.3  COI  3: 

Does  the  AMASS  provide  the  controls  and  displays  necessary  for  the  controller/supervisor  to 
perform  the  assigned  duties? 


6.3.1  COI  3  Results. 

The  following  issues  are  associated  with  this  COI: 


AT  Issue  6a:  (Medium)  AT  Training  should  include  a  clear  and  concise  presentation  on  the 
AMASS  Alert  Scenarios. 

Resolution:  Provide  a  video  demonstrating  each  AMASS  alert  scenario  on  the  potential 
collisions. 
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Status:  Currently  the  training  does  provide  slides  on  each  alert  scenario,  but  the  OT  test  team 
members  want  to  see  a  video  demonstration  on  each  AMASS  alert  scenario  (Visual  Aid  on  the 
general  timing  and  distance  of  each  alert  scenario). 


AT  Issue  7b:  (Medium)  The  AT  Users  Manual  provides  too  much  information  for  use  by  a 
controller  on  a  day-to-day  basis. 

Resolution:  Include  a  Quick  Reference  Card  to  summarize  the  AMASS  commands  (in  addition 
to  the  planned  Mini-Display  Control  Unit  (DCU)  keypad  template.) 

Status:  AMASS  Work  Group  has  this  issue  and  is  in  the  process  of  deciding  whether  or  not  to 
develop  a  quick  reference  card. 


6.4  COI 4: 

Does  the  AMASS  provide  the  controller  the  capability  to  inhibit/re-inhibit  specific  targets  from 
activating  alerts? 


6.4.1  COI  4  Results. 

No  concems/issues  were  expressed  with  the  inhibit/re-inhibit  functionality. 

6.5  COI  5: 

Does  the  AMASS  degrade  any  other  FAA  system  to  which  it  is  interfaced? 

6.5.1  COI  5  Results. 

The  following  issues  are  associated  with  this  COI: 

AF  Issue  10:  (Medium)  AMASS  is  generating  a  video  stitching  on  the  ASDE-3  Display. 
Resolution:  Modify  the  AMASS  equipment  to  resolve  the  video  timing  issue. 

Status:  A  screen  refresh  removes  the  video  stitching;  this  problem  will  be  fixed  (STR  has  been 
written). 


7.  CONCLUSIONS. 

The  Air  Traffic  (AT)  regression  testing  closed  30  out  of  37  issues  with  no  high  issues.  The 
remaining  seven  open  issues  contain  only  three  medium  and  four  low  priority  issues.  The 
medium  issues  are  in  section  6.  with  a  current  status.  The  Low  issues  have  been  passed  onto 
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Airway  Operational  Support  (AOS)  for  future  improvements  to  the  Airport  Movement  Area 
Safety  System  (AMASS). 


The  Airway  Facilities  (AF)  regression  testing  closed  17  out  of  19  issues  with  no  high  issues.  The 
remaining  two  open  issues  are  classified  as  medium.  The  medium  issues  are  in  section  6.  with  a 
current  status. 


The  Operational  Testing  on  the  AMASS  Critical  Operational  Issues  (COI)  questions  have  been 
evaluated/assessed,  resulting  in  no  high  issues.  The  AF/AT  Operational  Test  Team  members  felt 
that  the  AMASS  system  has  shown  dramatic  improvements  and  is  currently  operationally  ready. 


8.  RECOMMENDATIONS. 


Implement  a  System  bum  in  period  (~30  days)  at  each  site  to  verify  system 
adaptation/optimization  and  to  verify  system  performance  such  as  false/nuisance  alert  rate  in 
each  new  environment. 
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9.  ACRONYMS. 


AMASS 

Airport  Area  Movement  Safety  System 

AF 

Airway  Facilities 

AOS 

Airway  Operational  Support 

APS 

AMASS  Parameter  Study 

ARTS 

Automated  Radar  Tracking  System 

ASDE-3 

Airport  Surface  Detection  Equipment-3 

AT 

Air  Traffic 

ATL 

William  B.  Hartsfield  Atlanta  Airport 

AWG 

AMASS  Working  Group 

COI 

Critical  Operational  Issues 

DCU 

Display  Control  Unit 

DEN 

Denver  International  Airport 

DFW 

Dallas  Fort  Worth  International  Airport 

DT 

Developmental  Test 

DTW 

Detroit  Metropolitan  Wayne  Airport 

FAA 

Federal  Aviation  Administration 

LRU 

Line  Replacement  Unit 

MAOR 

Minimum  Acceptable  Operating  Requirement 

MOP 

Measures  of  Performance 

NAS 

National  Airspace  System 

OET 

Operational  Events  Test 

ORD 

Operational  Requirements  Document 

OT 

Operational  Test 

OUT 

Operational  Usage  Test 

RMS 

Remote  Maintenance  Monitoring 

SCIP 

Surveillance  Communications  Interface  Processor 

SFO 

San  Francisco  International  Airport 

STARS 

Standard  Terminal  Automation  Replacement  System 

STR 

System  Trouble  Report 

TAIU 

Terminal  Automation  Interface  Unit 

TRACON 

Terminal  Radar  Approach  Control  Facility 

HEARTS 

Enroute  Automated  Radar  Tracking  System 
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APPENDIX  A 


AT  OT,  MOP/MAOR  Result  Matrix 


TEST  DATE(S):  7/19/99  THROUGH  7/30/99 
Test  Coordinator:  Chuck  Dudas 
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Alerts  for  all  single-track  caution 
cells  for  tracks  on  closed  runways 
shall  be  inhibited  except  for  specific 
arrivals. 


Arrival-ASDE-3  Track  Data 
Correlation.  The  probability  of 
correct  arrival-ASDE-3  correlation 
shall  exceed  90  percent. 
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The  AMASS  shall  provide  the 
capability  to  establish  and  maintain 
all  of  the  site-specific  functional  and 
system  data  bases  to  adapt  the 
operations  of  AMASS  at  any  airport. 


PASS/  ' 

FAIL  ISSUE  PROPOSED  SOLUTION  PRIORITY 


A- 8 


This  capability  shall  be  available  at 
each  AMASS  site. 
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adaptation  capability  must  allow  for 
Source:  ORD  3.r.(9)  4-sided  segment  definition  for  each 

movement  surface. 
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function  that  can  be  activated  by 
depressing  the  “Enter”  key  or 
canceled  by  depressing  the  “ESC” 
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keypad  functions,  other  functions 
shall  be  controlled  via  the  ASDE-3 
keyboard  using  function  “hot  keys”  to 
activate  the  function  when  the 
keyboard  is  in  AMASS  operation. 
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(b)  If  there  is  no  saved 

configuration,  a  data  base 
configuration  that  has  been  defined  as 
the  default  startup  configuration. 
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operating  Terminal  Automation 
Interface  Unit  (TAIU). 
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A  combination  of  mode  and  alert 
inhibit  indicators  shall  be  displayed  if 
the  situations  exists  simultaneously. 


MOP-3.27;  MAOR-3.27:  The  AMASS  dynamic  PASS 

presentation  must  include  hold  bars 
Source:  ORD  3.d.(3).(a)  at  actjve  rUnway  intersections. 
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and  remain  displayed  as  long  as  the 
alert  condition  continues. 
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numeric  and  alphabetic  identifiers  of 
runways  and  taxiways, 
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The  statistical  analysis  package  shall 
provide  a  hard  copy  print  out  and  an 
electronic  text  file  containing  the 
following  information: 
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In  addition,  the  statistical  analysis 
package  printed  report  shall  include  a 
summary  of  the  number  of 
occurrences  that  each  cell  generated 
an  alert  during  the  log  period. 
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Modifications  to  the  displayed  alert 
list  shall  be  made  on  a  complete  radar 
scan  basis, ... 


1 1 
ct  qz 
<D  ^ 
^  £ 
‘5 


rt 


x:  £ 
w  O 


00 
00 

<  t 

S  <2 

<  .£ 

«  XJ 

fS  g 


1  B 

O  XJ 
fc  (L> 
g-  T3 

S’  C 

ca 

OO  c 
_  O0  *~ 
'j=  <  «> 


<U 


■a  *£ 

g  £ 
£  2 
d>  a> 
x>  XJ 


o 

u 

E 

o 


<D 

Q 


0.4  E 


O 

u 

E 

o 


00 
a>  £ 

3l 

s  < 

<£  <D 

2  -5 

|| 

c  CL 
<u  r, 

C/J  S 
<U 

*•:§ 

c3  «2 

2  '5  r 
«  § 
«1>  W5 

~  fe  .2 
^  73  £ 


O 

D 

E 

o 


00 

00 

< 


A-29 


controls  to  operationally  control  the  controller/supervisor), 
system? 
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keys”  along  with  the  trackball  shal 
be  used  to  identify  the  desired 
selective  aircraft  filtering  function 
and  track(s). 
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warn  of  impending  situation. 

Aircraft  to  far  out  to  issue  go  around.  Could  have  gotten  aborted 
a/c  off  the  runway  .  If  not  watching  this,  the  alert  would  have 
been  effective  to  warn  of  an  impending  situation. _ 
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Case  19cr  Cell  l;l>l.c?  Departure  Chasing  Departure 

When  following  the  Draft  ATC  AMASS  Operational  Procedures,  how  does  the  Test  team  responses: 
AMASS  response  affect  your  ability  to  perform  your  assigned  duties? 

...  Not  tested... 
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APPENDIX  J 

AT  OT,  OUT  Sub-Test  Questionnaires  at  ATL 

(July  1999) 


for 


J-l 


J-2 


APPENDIX  K 


AT  Regression  Questionnaire  Test  Responses 

(June  2000) 
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I  Questionnaire  AT-ORT-3,  UIT,  Changing  Operational  Configurations 
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K-5 


Questionnaire  AT-ORT-3,  UIT,  Training  Documentation,  SHEET  2 


APPENDIX  L 


AF  OT,  MOP/MO AR  Result  Matrix 


Test  Date(s):  _ February,  2000 
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MAOR-5a5:  One  additional 
connection  capability  shall  be 
provided  for  an  additional  unit  (PC 
platform,  keyboard  and  mouse). 


PASS/  ISSUE  PROPOSED  SOLUTION  PRIORITY 

FAIL 
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APPENDIX  M 


AF  OT,  AF  Questionnaires  (February  1999) 


AFO-2,  AF  Questionnaires 


AMASS  LRU  Name:  RF 

Filter  Box  Assembly,  PC  Platform 
extender  transmitter  and  receiver, 
Fixed  hard  drive,  Removable  disk, 
Voice  processor,  Amp/speaker, 
Computer  Interface  module,  PC 
platform,  Modem,  Micro-bus 
interface,  Power  monitor,  Display 
processor  interface  card, 

Recommendations. . . 

a.  Technician  #1 

b.  Technician  #2 

c.  Technician  #3 

Q 1 LRU  Replacement 

Was  the  LRU  removal  and 
replacement  procedure  described 
in  a  suitable  and  efficient  manner? 

a.  Redundant  and  takes  a  lot  for  granted.  Removal  of  some  items  such  as  pins 
(spring  loaded)  not  very  descriptive. 

Q2.-  AMASS  Maintenance 

Section 

Is  the  AMASS  Maintenance 
section  detailed  and 
comprehensive  enough  to 
troubleshoot  and  fault  isolate,  to 
the  defective  LRU? 

a.  Fault  flow  diagrams  and  equipment  names  don’t  necessarily  follow  block 
equipment  names,  very  hard  to  follow  and  find  specifics  such  as  equipment  names 
in  tech  manual. 

b.  Fault  logic  diagram  needs  to  indicate  a  figure  number  to  return  to  on  Figure  7-5. 
See  blocks:  Perform  AC&DC  power  fault  isolation 

Q3.-  AMASS  Maintenance 

Section 

Is  the  AMASS  Maintenance 
section  detailed  and 
comprehensive  enough  to  assist 
you  in  physically  locating  the 
LRUs? 

a.  Definitely  not,  equipment  LRU  names  are  not  consistent  throughout  tech  manual 

b.  Site  selected  installation  of  speaker/amplifier  assembly. 

Q4.-  AMASS  Maintenance 

Section 

a.  Redundancy.  Equipment  very  hard  to  access  if  equipment  cabinet  is  located 
against  the  wall. 

Is  the  AMASS  maintenance 
section  detailed  and 
comprehensive  enough  to  assist 
you  in  replacing  the  LRUs? 

M-l 


Q5.-  Training  Effectiveness 

a.  Still  could  use  OS-2  and  Windows  95  overview  for  file  management. 

Was  the  training  provided  to  you 
(system  overview,  manuals, 
system),  sufficient  to  use  the 
AMASS  system  in  an  operational 
environment? 

b.  Will  be  attending  equipment  school  in  OKC  when  available. 

Q6.-  System  Setup  Training 

Effectiveness 

a.  Data  collection  and  clutter  map  collection  not  expressed 

Was  the  training  provided  to  you, 
sufficient  to  use  the  AMASS 
manuals  and  system  to  perform  a 
setup,  data  collection,  or 
calibration  on  this  AMASS 
system? 

Q7.-  AMASS  Software  User’s 

Manual 

Is  the  AMASS  Software  User’s 
Manual  detailed  and 
comprehensive  enough  to  assist 
you  in  the  operation  and  setup  of 
the  AMASS  system? 

a.  I’m  not  a  so  called  expert  in  this  phase,  so  I  really  can’t  comment 

a.  Table  7-1,  which  was  referenced  by  the  flow  chart  troubleshooting  referred  to 
AMI-Diagnosis.  Where  is  this  found? 

Q8-  BIT/FIT 

a.  Not  necessarily 

Was  the  Built-in-test  (BIT)  and 
Fault-isolation-test  (FIT) 
sufficient  in  providing  guidance  to 
the  faulty  LRU? 

b.  With  the  PC  expander  (Rx)  broken,  the  maintenance  terminal  is  blank.  You 
must  depend  on  the  fault  logic  diagram. 

Q9.-  Support  Equipment 

a.  Absolutely  not 

Were  you  informed  of  the  support 
equipment  requirements  for 

AMASS  LRU  removal  and 
replacement  before  beginning  the 
maintenance  task?  (Tools, 
manuals,  etc.) 
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Q10.-  Overall  System  Safety  a.  Sometimes  it  tells  you  to  pull  the  plug  on  the  208V.  Pulling  the  plug  on  the 

1 15Vdisabling  the  light,  which  is  needed. 

Do  the  test  tools,  AMASS 

documentation,  system  or  a.  All  procedures  should  state  unplugging  208VAC  from  cabinet.  Unplugging 

procedures  allow  you  to  perform  1 15 VAC  removes  just  the  light  (which  is  needed)  and  the  receptacles  at  the 

this  LRU  removal  and  base  of  the  equipment,  which  may  be  needed  for  test  equipment, 

replacement  in  a  safe  manner? 

b.  Procedure  has  you  secure  power  at  the  A/C  power  switch  on  the  front  panel. 
The  procedure  should  have  you  secure  power  by  unplugging  cabinet  or  secure 
circuit  breaker. 

Additional  comments/suggestions  regarding  AMASS  LRU  removal  and  replacement: 

a.  Removal  and  replacement  of  LRUs  are  very  unsatisfactory  in  an  environment  where  the  AMASS  cabinet  is  located 
against  the  wall.  Use  of  a  mirror,  body  contortions,  feeling  method  and  removal  of  unnecessary  LRUs  to  gain 
access  to  the  failed  unit  is  very  time  consuming  and  can  be  somewhat  destructive. 

b.  No  comments 

c.  Documentation 

The  AMASS  troubleshooting  charts  did  not  lead  me  to  the  cause  in  any  of  the  four  problems  I  was  given  to  solve.  I  was 
never  directed,  by  the  charts,  to  observe  either  the  RMST  AMASS  inquiry  screens,  the  AMASS  application  screen  for 
status,  or  POST  FIT  results.  In  fact,  I  was  led  to  unnecessary  removal  and  replacement  of  components  several  times. 
Only  after  loosely  interpreting  the  charts,  was  I  once  led  to  run  FIT  which  gave  the  correct  result  (with  exception,  see 
note).  Before  starting  each  of  the  troubleshooting  problems,  I  made  quick  cursory  observations  of  the  ASDE  and 
AMASS  screens  and  was  able  to  correctly  determine  the  probable  cause  in  each  case.  The  charts  actually  caused 
extended  outage  and  I  personally  would  have  abandoned  their  use  after  the  first  few  moments  in  favor  of  my 
independent  observations.  There  were  instances  of  contradiction  as  when  the  AMASS  display  indicated  an  LRU 
(DP1/DM1)  had  passed  POST  (or  in  some  cases,  displayed  blank  status  field)  but  failed  the  FIT  results  returned  to  the 
RMST  (this  being  the  true  condition).  Running  FIT  manually  did  not  remedy  the  situation  nor  did  replacing  DPI/DM  1 
as  suggested. 

Note:  Found  AMASS  TAIU  comm  fault  Ran  FIT,  which  correctly  identified  the  modem  as  the  defective 
component  The  modem  was  replaced  and  operated  normally.  Ran  FIT  again  to  verify  operation.  FIT 
erroneously  returned  a  failure  with  probable  causes  of  I)  serial  card ,  2)-voice  processor ,  and  3)-comp  i/f 
Ran  FIT  one  more  time  with  no  failure  found. 

The  Table  of  Contents’  is  incomplete.  Several  LRUs  (all  of  the  A2  Card  Rack  Assembly  CCAs)  were  not  listed  in  the 
removal  and  replacement  procedures.  Other  maintenance  procedures  were  difficult  to  find  and  usually  required 
browsing  the  List  of  Illustrations  for  the  LRU  with  the  assumption  that  the  required  procedure  would  be  located  near  the 
figure.  Incorporating  a  simple  index  in  the  documentation  would  greatly  enhance  efficiency  in  field  operations. 

Hardware 

Access  from  the  front  of  the  cabinet  for  the  CPU,  video  switch,  and  modem  is  nearly  impossible.  Replacement 
procedures  were  crude  and  cumbersome.  For  example,  the  video  switch  must  be  unsecured  and  rested  on  the 
unprotected  (open  cabling  and  backplane)  units  below  to  accommodate  CPU  chassis  removal.  Cable  routing  further 
complicated  the  procedures  as  there  was  barely  enough  length  to  accommodate  the  re-connection  of  the  cables.  These 
situations  can  easily  contribute  to  component  damage.  Re-positioning  of  the  modem  to  the  very  top  of  the  rack  with  the 
video  switch  next  and  the  CPU  at  the  lowest  available  space  could  elevate  an  impossible  task  to  a  difficult  one.  Cabling 
needs  serious  attention  -  especially  for  ‘front  access  sites  only’. 

Software 

The  command  ‘ Sysres  -Amass’  is  only  valid  while  the  application  is  running  in  ‘ Operational  Mode  ’  but  initiating  the 
command  at  the  RMST  returns  ‘ System  Command  Processed ’  regardless  of  Amass  status.  This  invalid  response  appears 
to  occur  with  all  RMST  generated  commands.  A  second  example  would  be  the  command  Amass  -Online’  entered 
before  the  FIT  results  have  been  returned  to  the  RMST  -  the  command  was  ignored  (unbuffered)  and  a  misleading 
* System  Command  Processed’  message  was  displayed. _ 


M-3 


AMASS  Maintenance 

Procedures:  Dedicated  Clutter 
Collect,  Load  recorded  log  file, 
Load  current  log  file,  Data 
reduction  utilities,  Site-specific 
file  back-up  procedures 

Recommendations/The  problem  is... 

Q  L-  AMASS  Maintenance 

Procedures 

Is  the  training  and  AMASS 
manuals  detailed  and 
comprehensive  enough  to 
provide  you  with  a  functional 
and  operational  overview  of  the 
AMASS  system? 

a.  Picture  with  no  written  procedures. 

a.  No  written  procedure,  though  the  program  seemed  user  friendly  although  could  not 
load  current  log  file 

Q2.-  AMASS 

Documentation 

Is  the  AMASS  documentation 
detailed  and  comprehensive 
enough  to  allow  you  to  perform 
routine  maintenance  tasks  on 
the  AMASS  system? 

Q3.-  AMASS  Configuration 

Is  the  documentation  and 
training  of  sufficient  detail  to 
allow  you  to  perform  an 
operational  configuration 
change?  (Workstation  mode 
change,  load  recorded  log  file, 
inject  synthetic  tracks,  etc.) 

Q4.-  Dedicated  Clutter 

Collect 

Is  the  Dedicated  Clutter  Collect 
procedure  described  in  a 
suitable  and  appropriate 
manner? 

b.  Many  references  are  incorrect.  Some  of  the  steps  are  confusing  and  the  procedure 
ends  before  instructing  you  to  return  thresholds  to  normal  and  restoring  normal 
operations 

Q5.-  Support  Equipment 

Were  you  informed  of  the 
support  equipment  requirements 
for  the  AMASS  maintenance 
before  beginning  the 
maintenance  task?  (Tools, 
manuals,  etc.) 
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Q6.-  AMASS  Maintenance 
Procedures 

Is  the  AMASS  maintenance 
section  detailed  and 
comprehensive  enough  to 
troubleshoot,  isolate,  and 
reconfigure  the  system  upon 
locating  a  fault  in  the  Display 
Processing  Unit  (DPU)  cabinet? 


Additional  comments/suggestions  regarding  AMASS  Maintenance  Procedures: 

a.  Write  procedure  (step  by  step)  for  loading  recorded  log  file  for  playback. 

b.  Write  procedure  for  renaming  and  saving  current  log  file  for  access.  Consider  file  management  training 

c.  While  in  1  test: on'  mode,  all  synthetically  generated  targets  and  alarms  are  also  present  in  the  tower  cab  and  manual 
suppression  of  each  display  and  aural  alarms  is  necessary.  While  this  is  an  undesirable  situation,  it  is  workable  but 
should  at  least  include  a  cautionary  statement  in  the  procedure. 

There  were  no  detailed  contiguous  procedures  for  data  playback  and  required  searching  the  documentation  and  several 
empirical  iterations  to  accomplish  the  basic  task. 

Invoking  the  ' AMASS  MT  mode  (amass  - disconnect ),  causes  an  ASDE  fault. 

No  scenario  for  arrival  alarms  other  than  real-time  arrival  target  with  stopped  synthetic  target. 

There  are  no  site  specific  file  or  critical  file  backup  procedures. 

There  are  no  data  reduction  utilities. 

Incomplete  and  incorrect  clutter  map  procedures. 
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Terminal  Automation  Interface  Unit  (TAIU)  Comments 

Technician  a.  Most  LRUs  were  more  or  less  obvious  to  what  needed  to  be  removed. 


b.  The  two  days  spent  getting  familiar  with  the  TAIU  were  sufficient  to  get  an 
overall  understanding  of  the  functionality.  Troubleshooting  flow  chart  (page  C-3 
of  8)  refers  to  ARTS  interface  (primary/secondary  IOP  channel).  At  this  location, 
the  radar  technicians  don’t  work  on  the  ARTS. 


c.  OJT  training  was  adequate  in  scope  but  should  be  limited  to  2  or  3  persons  per 
session.  I  was  not  able  to  see  the  displays  or  controls  most  of  the  time  because  of 
the  number  of  people  gathered  in  the  limited  space  of  the  ARTS  environment. 

The  TAIU  troubleshooting  charts  were  acceptable  and  I  was  taken  to  the  correct 
result  each  time.  Consideration  should  be  given  to  switching  IOP  interfaces  to  aid 
the  process.  The  modem  configuration  procedure  addresses  only  ‘Idle  or 
Training’  and  not  the  ‘Online’  condition.  There  were  obvious  places  in  the  charts 
where  paragraph  references  would  have  been  very  useful. 


TAIU  Documentation 

The  flow  chart  for  IOP  repeater  amplifier  needs  to  have  a  couple  of  blocks  before  the  block  ’’Call 
for  assistance.” 

Add  a  block  to  check  ODS  for  IOP  configuration. 

Question:  Configured  ok? 

If  the  answer  is  Yes  -  Change  to  secondary  IOP. 

Question:  Are  FPU  data  blocks  now  present? 

If  the  answer  is  Yes  -  Interchange  IOP  repeaters  to  isolate  faulty  unit. 

If  the  answer  is  No  -  Reconfigure  ODS  correctly. 

Question:  Are  FPU  data  blocks  now  present? 

If  the  answer  is  Yes  -  the  problem  is  solved. 

If  the  answer  is  No  -  then  you  return  to  the  previous  block  where  the  ODS  is  configured 
properly  and  proceed. 

If  the  answer  is  No  after  changing  to  the  secondary  IOP,  this  is  where  the  Cal  for 
assistance  block  goes. 

Make  very  clear  whom  the  technician  is  to  call. 
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APPENDIX  N 


AF  OT  Issue  Result  Matrix 


AMASS  AT  OT  ISSUE  LIST 
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Box  assembly  instructs  you  to  secure  power  at  the  a/c 
power  switch.  Power  needs  to  be  secured  at  the  circuit 

_ breaker  or  by  unplugging  the  cabinet. _ 

Video  Stitching  AMASS  is  generating  a  video  stitching  on  the  ASDE-3  FAIL  Modify  the  AMASS  equipment  to 

Display.  resolve  the  video  timing  issue. 
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AF  OT,  AF  Regression  Comments 
(June  2000) 


AF  Comments  to  the  AMASS  Manuals 


I>  Comments  to  the  AMASS  TIB 


1 .  Problem:  The  format  of  the  AMASS  instruction  book  title  page  is  not  clear. 

Solution:  Reverse  the  title  page  format 

a.  Move  AMASS  Addendum  to  top  of  page  in  same  bold  print  as  ASDE 

b.  Move  the  ASDE  where  the  AMASS  is  now  and  use  the  smaller  print. 

2.  Problem:  TABLE  OF  CONTENTS  -  Very  hard  to  find  an  item  quickly.  Need  to  make  the  various 
sections  standout.  Also  the  Technicians  were  having  problems  looking  up  detailed  information  in  the 
AMASS  TIB. 

Solution: 

a.  Add  an  A  to  Volume  numbers  i.e.  Volume  7A 

b.  Separate  Section  and  Sub-Section  Titles  with  a  space  at  top 

c.  Capitalize  Section  and  Sub-Section  Titles 

d.  Use  indentation  under  Section  Titles  and  Sub-Section  Titles 

Example: 

3.0  OPERATION 

3.1  POWER  SYSTEM  UP 

3.2  CONTROLS  AND  INDICATORS 

3.2.1  Cabinet  Controls  and  Indicators 

3.2.2  Operational  Display  Unit  (ODU) 

4.0  POST-OPERATION 

e.  One  possible  solution :  Creating  A  SEPARATE  INDEX  VOLUME  may  help  the  technicians 
to  find  an  item  quickly. 

3.  Problem:  No  Boot  Sequence  for  AMASS/ASDE  in  the  AMASS  TIB  Manual  could  be  found.  If  ASDE 
is  powered  down,  what  is  the  sequence  of  events,  do  you  power  down  AMASS,  then  power  up  ASDE 
first  or  visa  versus.  This  information  is  contained  in  the  change  pages  for  the  ASDE  but  should  also  go 
into  the  AMASS  TIB.  Should  this  be  located  in  a  AMASS  TIB  flow  chart  diagram? 

4.  Overall  Comment:  The  AMASS  TIB’s  should  clearly  defined  in  the  procedures  when  the  Technician 
should  use  AMASS  MT  or  the  ASDE  MT  with  the  appropriate  commands  for  AMASS  disconnect. 

5.  Problem:  POWER  DOWN/POWER  UP  -  missing  steps  in  the  procedure. 

Solution:  Last  step  should  be  added  “Reset  -System”.  This  is  needed  to  clear  the  stored  faults 

6.  Problem:  COPY  LOG  FILES  -  page  7.10.50  -  routine  did  work, 

Solution :  Add  to  the  first  step  “FROM  ASDE  MT  TYPE:  AMASS  -DISCONNECT” 

7.  Problem:  PLAYBACK-  routine  did  work 
Solution : 

a.  Path  must  be  specified  where  to  put  it 
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b.  Instructions  do  not  tell  how  to  get  out  of  playback.  Add  step  to  say  “go  to  system  -  select  mode, 
select  operational” 

8.  Problem:  DEDICATED  CLUTTER  COLLECT  -  routine  did  work 
Solution: 

a.  Page  7.10.61,  “Menu  Path:  System>  Dedicated  Clutter  Collect”  -  the  word  “system”  (AMASS 
or  ASDE?)  keeps  changing  throughout  the  book.  This  needs  to  be  made  uniform. 

b.  Page  7.10.62  -top  sentence  -  remove  “weather  conditions”  or  “off  more  than  one  hour.”  Be  more 
specific:  If  there  is  no  reason  for  field  personnel  to  perform  this  procedure.  Then  we  should 
consider  stating  that  because  this  requires  power  down/power  up,  it  is  a  big  interruption. 

c.  Command  “AMASS-DISCONNECT”  causes  AMASS  fault  but  there  is  no  fault.  There  should  be 
a  difference  noted  on  the  screen  between  a  keyboard  command  to  disconnect  and  an  actual  fault 
that  causes  a  disconnect 

d.  Page  7.10.62  -  AMASS  Terminal,  under  Step  2  -  add  sub  step  c.  to  read  “There  is  a  short  wait 
for  automatic  clutter  window  to  appear.” 

e.  Step  5  should  reference  Shut  Down  Procedures,  Volume  1,  page  1.3.36,  paragraph  3.44 

f.  Step  6  Rewrite  to  read  “ASDE  needs  to  be  returned  to  facility  Area  12  Threshold  values  and  the 
IF  Thresholds  turned  “ON.” 

9.  Problem:  COPY  BACKUP  SITE  DATA/  DATABASE  FILES  -  page  7. 10.215,  When  copying  the 
DTW_DATA  Folder  over  to  the  F  drive  there  was  NOT  enough  capacity. 

Solution:  Create  a  separate  folder  for  each  the  Site  Configuration  files  and  the  Log  Files  or  write  a 
script  file  to  copy  only  the  site  configuration  files. 

10.  Faulted  speaker  amplifier,  disconnected  cable  at  the  top  of  AMASS  cabinet  -  Maintenance  Mode 

1.  Problem:  Figure  7-1,  troubleshooting  flow  chart  assumes  that  the  AMASS  is  connected  to  the 
ASDE  RMST  after  rebooting  the  AMASS  system. 

No  resolution  required:  In  AMASS  Configuration  file  the  following  line  was  commented  out: 

Control_Mode  =  MT 

MT  =  AMASS  MT  Only 

RMS  =  Connect  AMASS  to  ASDE  RMS 

therefore,  the  AMASS  would  not  auto  connect  to  the  ASDE  RMST  on  reboot.  During  normal 
operations,  the  AMASS  will  auto  connect  to  the  ASDE  RMST  on  reboot. 


2.  Problem:  The  technicians  thought  if  you  incorporate  the  “check  AMASS  BIT  results  on  the 
ASDE  MT  Inquiry  screen  first”  into  the  Troubleshooting  flow  charts.  This  could  find  the 
problem  immediately.  Then  run  the  Fit  at  the  ASDE  MT. 

Solution:  FIT  runs  a  more  detailed  fault  testing  then  BIT  provides,  solution  is  to  run  FIT  each 
time.  A  detailed  note  in  the  AMASS  TIB  Manuals  and  the  training  class  should  stress  the  use  of 
running  FIT  instead  of  relying  on  BIT. 

Side  Note  -  FIT  should  always  be  run  after  a  clean  boot,  if  you  just  run  FIT  (no  AMASS 
Reboot)  then  FIT  program  will  use  the  previous  history  and  may  provide  the  wrong 
information . 
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3.  Problem:  Technicians  were  having  a  difficult  time  following  the  flow  charts,  when  ask  to  return 
back  to  Figure  7-1. 


II  COMMENTS  TO  TAIU  TIB 

1.  No  procedure  in  TAIU  TI  manual  Section  5.2.e  to  set  the  system  clocks.  This  has  been  designated  as  a 
certification  requirement  in  (AF  Certification  Procedures)  Section  1 ,  Performance  Checks,  41  l.d.  1 . 

2.  Section  5.2.e  maintenance  checks:  reference  illustrations/pictures  of  the  various  systems  when 
available. 

m  COMMENTS  TO  THE  CERTIFICATION  PROCEDURES  6330.5A 
A.  TAIU 

1.  Section  1,  Performance  Checks,  41  l.d.lchange  paragraph  5.2.a  to  5.1. 

2.  Section  1 ,  Performance  Checks,  41 1  .d.  1  change  paragraph  525  to  528 

3.  Section  11,  TAIU,  396,  change  paragraph  525  to  528 

4.  Paragraph  528,  Checking  the  Altimeter  -  omit  calling  ATC  to  get  an  altimeter  reading  from  Step  5. 

This  will  be  an  annoyance  to  ATC  and  is  unnecessary.  The  best  procedure  is  to  check  the  DASI;  next 
best  is  to  check  the  DEDS  for  altimeter  reading 

5.  Section  1,  Performance  Checks,  412.i.3,  Computer  Power  Supply  -  change  paragraph  526  to  529 

6.  Paragraph  529,  Procedure  should  refer  technician  to  rear  of  cabinet  where  power  supplies  are  located. 
The  LEDs  are  on  the  power  supplies. 

Side  Note  3/2/2000 ,  DI  will  be  replacing  the  power  supplies  in  the  TAIU  Cabinet ,  and  the 
new  power  supplies  have  no  LEDs. 

7.  Paragraph  529  Step  B,  Need  to  make  reference  to  Illustration  on  p.15  of  15,  Appendix  A  of  the  TAIU 
TI  Manual. 

8.  Section  1,  Performance  Checks,  414,  if  available  in  the  TAIU  TI  Manual  should  make  reference  to 
Illustration  showing  grills  and  filters 


B  AMASS 

1.  Section  1,  Performance  Checks,  410.b,  Change  maintenance  procedures  paragraph  from  527  to  530 

2.  Change  Paragraph  530.d  Step  1  &  12  to  “At  an  ODU  in  the  air  traffic  control  tower  cab,  go  to  ASDE 
main  Menu,  scroll  to  the  AMASS  function,  activate  the  AMASS  Main  menu.” 
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3. 


Add  a  statement  to  Paragraph  530.d  Step  1 1  on  “AMASS  may  not  alert  immediately  due  to  the  amount 
of  time  required  for  AMASS  to  process  the  closure  of  a  runway.” 

4.  Change  Paragraph  530.d  Step  21 :  replace  “active”  to  “open”.  Change  “an  X”  to  “No  X”  and  change 
“each”  to  “either”. 

5.  Add  Step  22  “To  toggle  to  ASDE  Menu  -  Depress  Function  -  U  -  Enter,  then  Scroll  to  Exit  -  Enter”, 

6.  Section  1,  Performance  Checks,  412.h,  Quarterly  -  add  step  3,  “Check  fans  on  PC  Platform.” 

C.  ASDE 


Page  3-2,  paragraph  311  to  317,  ASDE  Handbook  manual  paragraph  references  are  correct,  but  the  Site 
Manual  has  not  been  updated.  Why  two  references? 

D.  Appendix  I  Certification  Requirements,  Table  1 

1 .  SIA  Incursion  Alert  Test  -  change  reference  paragraph  527  to  530 

2.  Acronym  TSGDT  Description  is  wrong,  needs  to  be  changed  to  TSSDD  throughout  table. 
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